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Abstract. In this paper, we show the moduU spaces of stable sheaves on K3 surfaces are irreducible 
symplectic manifolds, if the associated Mukai vectors are primitive. More precisely, we show that they are 
related to the Hilbert scheme of points. We also compute the period of these spaces. As an application of our 
result, we discuss Montonen-Olive duality in Physics. In particular our computations of Euler characteristics 
of moduli spaces are compatible with Physical computations by Minahan et al. 



0. Introduction 

0.1. Main result. Let X be a projective K3 surface defined over C and H an ample divisor on X. Let 
u be the fundamental class of X. Let E' be a coherent sheaf on X. By the identification H^{X^ Z) = Zoj, 
we regard the second Chern class C2{E) as an integer. Since (ci(£')^) is even, the second Chern character 
ch2(-E) belongs to Z. We define the Mukai vector of E by 

viE):^ch{E)Vtd^ 

= Tk{E) + ci{E) + {Tk{E) + ch2iE))u; e H*{X,Z), 

where we identify H'^{X, Z) with Z and tdx = 1 + 2a; is the Todd class of X. For an element v E H*{X, Z), 
we denote the 0-th component vq G H^{X,'L) by rku and the second component vi e H'^{X,Z) by ci{v). 
We set i{v) := gcd(rkD, ci(u)) G Z>o. Then v is written as f = ^(w)(r-|-^) + auj, where r € Z, ^ € H^{X,'Z) 
and r + ^ is primitive. We denote the moduli space of stable sheaves E of v{E) = v hy Mff{v). If v is 
primitive and H is general in the ample cone Amp(X) of X (i.e. there are at most countable number of 
hyperplanes Wn C H^{X,Q), n = 1,2, .. . which depends on v and H belongs to Amp(X) \ U„M^„ [Y3]), 
then Mh{v) is a smooth projective scheme. In [Mul], Mukai showed that Mh{v) has a symplectic structure. 
In order to get more precise information, Mukai [Mu2] introduced a quite useful notion called Mukai lattice 
(H* {X, Z) , ( , ) ) , where the pairing is defined by 

{x,y) := - / x'^y 

■'^ (0.2) 

{xiyi - xoy2 - X2yQ), 

ix 

where Xi G H^^{X, Z) (resp. yi G H^^{X, Z)) is the 2i-th component of x (resp. y) and a;^ = xq — Xi + X2- 
Hence ( , ) is an integral primitive bilinear form on H*(X, Z). By the language of this lattice, we can write 
down Ricmann-Roch theorem in a simple form: We set 

2 

X{E,F) ^dimExt'(£;,F) (0.3) 

i=0 

for coherent sheaves E and F. Then Ricmann-Roch theorem implies that 

x{E,F) = -{v{E),viF)). (0.4) 

In particular we get that dim Mh{v) = {v^) + 2. 

If u is a primitive isotropic vector, then Mh{v) is a surface with a symplectic structure. Mukai proved 
that Mh{v) is a K3 surface and described the period in terms of Mukai lattice. If u is a primitive Mukai 
vector of (u^) > 0, then Mh{v) is a higher dimensional symplectic manifold. If rku = 1, then Mh{v) is 
the Hilbert scheme of points on X. Indeed every torsion free sheaf of rank 1 is give by Iz <E) L, where Iz 
is the ideal sheaf of a 0-dimensional subscheme of X and i is a line bundle of ci{L) = ci{v). Beauvillc 
[B] proved that it is an example of higher dimensional irreducible symplectic manifold. For an irreducible 
symplectic manifold, Beauville [B] defined the period and proved local Torelli theorem. As an example, 
he also computed the period of Hilbert scheme of points on X. For higher rank cases, Mukai [Mu3] (rank 
2 case), O'Grady [01] {i{v) = 1 case) and the author [Y5] ((w^) > 2i!(u)^ or £{v) = 1 case) proved that 
Mh{v) is an irreducible symplectic manifold and described the period of Mh{v) in terms of Mukai lattice. 
For classification of Mh{v), it is important to determine the period. Indeed, it is a birational invariant 
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([Mu3]), and afRrmative solution of Torelli conjecture will imply that an irreducible symplectic manifold is 
determined by its period, up to birational equivalence. 

In this paper, by using [Y5] extensively, we prove the following theorem, which is expected by many people 
(for example, see [D], [Mu3], [01]). 

Theorem 0.1. Let v be a primitive Mukai vector such that rku > and ci{v) G NS(X). 

(1) Mh{v) is not empty for a general ample divisor H if and only if (w^) > —2. 

(2) Assume that (w^) > —2. Then for a general ample divisor H , 

(2-1) Mh{v) is obtained by compositions of deformations and birational transformations fromYiiVo"^ 

In particular Mh{v) is an irreducible symplectic manifold. 
(2-2) Let Bmh(v) be Beauville's bilinear form on H'^{Mh{v),Z). Then 

is an isometry which preserves Hodge structures for (u^) > 2, where 6y : ^ H'^{Mh{v),'Z) is 
the canonical homomorphism defined by using a quasi-universal family. 

Here we only use deformations of Mh{v) induced by deformation of complex structures of X. 
Since birationally equivalent Calabi-Yau manifolds have the same Hodge numbers ([Ba],[De-L]), we get 
the following Corollary. 

Corollary 0.2. Keep the notations as above. Then hP''^{MH{v)) = /i^'*(Hilb^ ^^^^^). In particular, 
X(MH(i;))=x(Hilbf >/'+^). 

In [V-W], Vafa and Witten considered a partition function Z^{t), a G H'^{X, Z), r e M := {z e C|3z > 0} 
associated with = 4 super symmetric Yang-Mills theory on a 4 manifold X. Under suitable vanishing 
conditions (e.g. H^{X,ad{E) (8) Kx) = 0), it is related to "Euler characteristics" of moduli spaces of vector 
bundles. For a K3 surface case, Z"{t) is given by 

ZnT)= "x(A/«(^;))"(Z^"'>/2^ (0.5) 

rk v—r 
ci {v)—a 

where "x{Mh{v))" is a kind of "Euler characteristics" of a suitable compactification of Mh{v). Recently, 
this invariant was computed in [MNVW]. In section 4, by using Corollary 3.2, we shall check that their 
computation coincides with the Euler characteristics of Mh{v), if v is primitive. 
For a non-primitive Mukai vector, we have the following existence condition. 

Corollary 0.3. Let v be a Mukai vector o/rku > 0. Then there is a semi-stable sheaf E of v{E) = v with 
respect to a general ample divisor H if and only if v = nw, n £ Zi, w (£ H*{X, Z) with (w'^) > —2. 

0.2. Outline of the proof. We shall explain how to prove (2-1) of this theorem. In [Y3], we discussed 
chamber structure of polarizations. Let v = l{r -\- -\- auj, ^ € NS(X) be a Mukai vector of I = £{v) and 
r > 0. We choose an ample divisor H on X which does not lie on walls with respect to v. Then 
(ti) for every ^-semi-stable sheaf E of v{E) = v, if F C E satisfies {ci{F),H)/ rkF = {ci{E), H)/ ikE, 
then ci{F)/TkF = ci(£')/rk£'. 

Thus v{F) = l'{r + ^) -|- a'u) for some Z',a'. In particular, if v is primitive, then Mh{v) is compact. Let 
M{v) be the stack of coherent sheaves E of v{E) — v. We shall fix a general ample divisor H with respect 
to V. M.{v)^'"^ (resp. A^(u)''*) denotes the open substack of Ai{v) consisting of /i-semi-stable sheaves (resp. 
/x-stable sheaves). 

In [Y5], we proved Theorem 3.1 under the assumption (u^) > 21^ or I = 1. Hence we may assume 
that (w^) < 2P and / > 1. However for convenience sake of the reader, we only use the results for I = 1 
case. We note that isometry group 0(i?*(X, Z)) of Mukai lattice acts transitively on the set Vn ■= {x € 
H*{X,'Z)\x is primitive, (a;^) = 2n} and 0(iJ*(X, Z))/±l is generated by the following 3 kinds of isometries: 

1. Translation: For N e Pic(X), 

Tn: H*{X,Z) ^ H*{X,Z) 
X ^ c\i{N)x 

is an isometry. 

2. 0(i?2(x,Z)) acts on 0(iJ*(X,Z)). 

3. Reflection: For a (-2)- vector vi G H*{X,'L), 

: H*{X,E) H*{X,Z) 

X 1-^ X -i- {x, Vi)vi 

is an isometry. 



(0.7) 



Therefore it is very important to understand reflections. 

Geometric realization of reflections: As we shall see in Corollary 3.3, a reflection is realized as a Fourier- 
Mukai transform. Here we shall explain a special case. Let Ei be a stable vector bundle of Ext^(i?i, Ei) = 
{El is called exceptional vector bundle). Since Ext {Ei,Ei) = Hom(i?i, = C, Riemann-Roch theorem 
implies that (w(-Ei), v{Ei)) = — x(-Ei, = —2. Thus vi := v{Ei) is a (— 2)-vector. Let i? be a stable vector 
bundle of v{E) — v. Assume that 

(a) ¥.yii\Ei,E) = 0, i = 1,2. 

(b) 4> : El® Hom(i?i, E) ^ E \s, surjective and kert/) is stable. 

Then w := v{ksv (p) is given by xi^i, E)vi — v — —iv + {v,vi)vi). Thus — u(ker(/)) is the (— 2)-reflection 
of V hy vi. Hence under conditions (a) and (b), (—2) -reflection of Mukai lattice induces a birational map 
Mh{v) • • • — > Mh{w). Replacing ker by coker((/)^ : E'^ E^ (g) Hom(i5i, i?)^), we may replace (b) by the 
condition (b'): 

(b') (f) : El® Hom(i?i, E) E \s surjective in codimension 1 and kerc/) is stable. 

Thus under (a) and (b'), we have a birational map Mh{v) ■ ■ ■ ^ Mh{w^)- We would like to apply this story 
for a suitable pair of vi and v which satisfy {vi,v) = —1. In order to get condition (b'), we shall prove the 
following key lemma which was proved under the assumption Ir < ri < (Z + l)r in [Y5, Prop. 4.5]. 

Lemma 0.4. Let {X,H) be a polarized smooth projective surface ofNS{X) = ZiJ. Let (ri,c?i) and {r,d) he 
pairs of integers such that ri, r > and dri — rdi = 1. We assume that Ir < ri. Let Ei he a pi-stable vector 
hundle o/rk(i?i) = ri and deg(£'i) = di, where deg(£'i) ~ {ci{Ei), H)/{H^). 

(1) Let E he a fi-stahle sheaf of rk(i?) = Ir and deg{E) = Id. Then every non-zero homomorphism 
if : El E is surjective in codimension 1 and ker Lp is a pi-stahle sheaf. 

(2) Let E' he a fi-stahle vector hundle o/rk(_E') — ri — Ir and deg(£") = di — Id. Let (j) : E' —t Ei he a 
non-zero homomorphism. Then 4> is injective and E := coker0 is a fi-semi-stable sheaf. 

For the proof of this lemma, we use the following fact: 

• We consider the triangle in with vertices (0,0), (ri — lr,di — Id) and (ri,c?i). Then there is no 
integral point in the interior of this triangle. 

Indeed, this condition gives a strong restriction on homomorphisms ip, (j) and the Harder-Narasimhan poly- 
gons of her if and coker^. The proof will be done in Preliminaries. 

In order to use this lemma, we need to compare A^(u)'^''^ and Ai{v)^^ . More precisely, we need dimension 
counting of various constructible substacks of A^(w)'""'. Technically this is the most important part in this 
paper. In [D-L], Drezet and Le Potier computed the dimension of the substack of non-semi-stable sheaves. In 
their computation, the existence of exceptional vector bundle is very important. In our case, we concentrate 
our consideration on A1(?;)'''"'. By our assumption on H, exceptional vector bundle E of v{E) = r + ^ + buj, 
& G Z is important. Hence we divide our proof into two cases: 

A. There is no (-2) vector of the form r + ^ + bcu, i.e. ((^^) + 2)/2r ^ Z. 

B. There is a (-2) vector of the form r + ^ + bcu, i.e. ((^2) + 2)/2r e Z. 

In section 2, we treat case A. In particular, we prove the following inequality: 

dimiMiv)"'" \ Miv)"') < {v^). (0.8) 



Then we can apply Lemma 'X4- For suitable choice of 

1. a primitive Mukai vector v := l{r dH) + au) on {X, H) and 

2. an exceptional vector bundle Ei of v{Ei) = ri + diH -\- aito, 

we can construct a birational map Mh{v) • • • Af//(w^) sending a general /i-stable vector bundle E G 
Mh{v) to F := coker(i;^ ^ Hom(£;i,£;)^ (g> E^) e MHiw"") where H and v' satisfy that (1) {H^)/2 = 
{ri{v'^)/2l + r)ri/l-r'^ > 0, (2) {vi,v) = -1 and (3) e{w'^) = 1 and hence Theorem [ojI holds for Mh{w'^). 



We remark that we n eed to choose a sufficiently large ri for the condition (1). In the same way as in [Y5, 4.3], 
we get Theorem for case A. More precisely, considering deformations of Mh{v) induced by deformations 
of {X, H) and translations T/v, we can reduce the problem to this situation. 

In section 3, we treat case B. If {v'^) > 2P, then we also have the inequality ( |0.8|) , and hence the same 
proof as in case A works. If (u^) < 2P, then it is known that there is no /z-stable sheaf. Hence we can not 



apply Lemma 0.4 in this form. When the (—2) vector is v{Ox) = 1 -f cd, T. Nakashima found the following 
fact: 

We set V = I — au!. Then the inequality < (i;^) < 2P implies that < a < L We assume that a > 2. Let 
be a /x-stable vector bundle ofv{E) ^a-luj. Then H° {X , E) =H'^{X,E) = and dim H^{X,E) = l-a. 
We consider the universal extension (another example of reflection) 

0^ E E' H\X,E)(g)Ox ^0. (0.9) 
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It is easy to see that E' is a stable vector bundle and we get an immersion Mfi{a — /u;)'^*''°'^ ^ Mh{v), 
where Mnia - 1^^'^°" is the open subscheme of Mff{a — lui) consisting of /i-stable vector bundles. 

This result can be easily extended to general cases. Hence what we should do is to prove the irreducibility 
of Mh{v) and the classification of Mh{v) consisting of non- locally free sheaves. By similar dimension 
counting as in case A, we shall classify non- locally free components and prove the irreducibility of Mh{v). 
The classification of non- locally free components of Mh{v) is described as follows: 



Proposition 0.5. Keep the notations in Theorem 0.1. Then Mh{v) consists of non-locally free sheaves if 
and only if rkv — I, v ; 

spaces, Mh{v) = Hilh^ 



and only i/rkw — 1, v — (rkwo)'yo ~ ^ or v = I ~ to, where vq is a Mukai vector of (vq) = —2. For these 



1. Preliminaries 

1.1. Notation. Except sections 1.5.1 and 1.5.2, we assume that X is a K3 surface. For a scheme 
PS ■ S X X S denotes the projection. For a Mukai vector v, we fix a general ample divisor H which 
satisfies {[\) in section 0.2. Obviously for any subsheaf E' C E oi /j,-semi-stable sheaf E of v{E) — v, if 
ci{E')/ TkE' = ci{E)/ rkE, then v{E') also satisfies (I]). M{v), M{v)'^'"' and M{v)'^'' are stacks in section 
0.2. M{v)^'' and M{v)^ denote the open substack of M{v) consisting of semi-stable sheaves and stable 
sheaves respectively. Since M{v)'^^'' is bounded, it is a quotient stack of an open subscheme of some quot- 
scheme by some general linear group (see Appendix). Hence our dimension counting of substack of 
can be regarded as that of subscheme of some quot scheme. Mh{v)^^ (resp. Af//(w)'°'^) be the open subscheme 
of Mh{v) consisting of /x-stable sheaves (resp. stable vector bundles). 

Mukai homomorphism: Let £ be a quasi-universal family of similitude p on Mh{v) x X, that is, £\{e]xx — 
E®P for all E £ Mh{v) ([Mu3]). By using £, Mukai constructed a natural homomorphism 

e,:v^^H^{MHiv),Z)f 

defined by 

Ov{x) ^ pMH(v)*{{ch£)^ 

where H'^{Mh{v),Z) f is the torsion free quotient of H'^{Mh{v),Z). We note that 0^, does not depend on 
the choice of a quasi-universal family. 



1.2. Some results from [Y5]. We collect some results which are necessary to prove Theorem p.l| . 

Theorem 1.1. Let v = l{r + £,) + auj, ^ G i?^(X, Z) be a primitive Mukai vector such that I = £{v) and 
r > 0. If {v'^)/2 > P or I = 1, and H is general, then Mh{v) is obtained by compositions of deformations 
and birational transformations from Jiilh^^ In particular, Mh{v) is an irreducible symplectic manifold. 

Let Bmh(v) be Beauville's bilinear form on H'^{Mh{v), Z). // (v^) /2 > P , or I = 1 and (u^) /2 = 1, then 

is an isometry which preserves Hodge structures for (v^) > 2. 

When I = £{v) — 1, this theorem was first proved by O'Grady [01]. In this paper, we only use this 
theorem for the case where £{v) — 1. 

The following is essentially due to O'Grady [01]. We can see a different proof based on Gottsche and 
Huybrechts' argument [G-H] in [Y7]. 

Proposition 1.2 ([Y5, Prop. 1.1]). Let Xi and X2 be K 3 surfaces, andletvi := l{r+^i)+aiLu e H*{Xi,Z) 
and V2 /(r + ^2) + i2^ G H* {X2,Z) be primitive Mukai vectors such that (1) r,l > 0, (2) r + ^i and r + £,2 
are primitive, (3) (vf) — (wl) = 2s, and (4) ai = 02 mod I. Then Mh^{vi) and Mh2(v2) oltc deformation 
equivalent. In particular, Mh^ (wi ) is an irreducible symplectic manifold and is an isometry of Hodge 
structures if and only if M {{2(^2) cind Oy^ have the same properties. 



Lemma 1.3 ([Y5, Lem. 5.1]). Let xi,X2,X3,yi,y2,y3 be integers such that xi, X2, X3 >0 and yix^ — Xiy^ 
1- If 



yi>yi>yi, (i.i) 

Xi X2 Xz 



then X2> xi + X'i. 
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Lemma 1.4 ([Y5, Lem. 4.1]). Let {X,H) he a polarized smooth projective surface o/NS(X) = Z_ff. Let 
(rijdi) and {r,d) he pairs of integers such that ri,r > and dri — rdi = 1. Let Ei he a ii-stahle vector 
bundle of rh{Ei) — ri and deg(i?i) = di, where deg(i?i) = {ci{Ei),H)/{H'^). Let E he a fi-stahle sheaf of 
rk(i?) = Ir and deg{E) = Id. Then the non-trivial extension 

O^Ei^E'^E-^0 (1.2) 

is a fi-stahle sheaf. 



Lemma 1.5 ([Y5, Lem. 4.4]). Let v be an arbitrary Mukai vector ofrkv > 0. Let A4/f (w)^** be the moduli 
stack of pL- semi- stable sheaves E of v{E) = v, and AAh{v)p^'^'' the closed suhstack of AAh{v)^'^'^ consisting 
of properly ^-semi-stable sheaves. We assume that (w^)/2 > P. Then 

coAimMH{vf^'' > {v^)/2l-l + l. (1.3) 

In particular, if is not empty, then there is a pi-stable sheaf E of v{E) — v. 

Proof. Since we need the proof of this lemma, we shaU give an outUne of the proof. For more details, see 
[Y5, sect. 5.3]. By Mukai [Mul], we get that 

dimMHiv f > {{v^) + 2) - 1. (1.4) 

We shall show that 

dimMnivf < i{v^) + 1) - i{v^)/2l -l + l). (1.5) 

For this purpose, we shall estimate the moduli number of Jordan-Holder filtrations. Let £^ be a /i-semi-stable 
sheaf of v{E) = v and let C Fi C F2 C • • • C F( — E h e a Jordan-Holder filtration of E with respect to 
^-stability. We set Ei := Fi/Fi^i. By using Lemma 5J in Appendix successively, we see that the moduli 
number of this filtration is bounded by 

^(diniExt^(£;j,£;,) - dimRom{Ej , Ei)) = ~xiE,E) + '^xiEj,E,) + '^dimExt'^ {Ej , E,) 1. 
i<j i>j i<j (1.6) 

We set v(E) := Ir + l^ + auj and v{Ei) := kr + + a^u}, where ^ e NS(X). Since {v{Ei),v{Ej)) = 
hlji^'^) — r{liaj + IjOi), we see that 

Y: x{E„E.,) = - ^(.(i?,), «(i?.)) = - E ^''''^t^""'^'^ - (1-7) 

i>j i>j i * 

We set maxi{li} = (/ — k). Let ig be an integer such that {v{Eig)'^) > 0. Since J2i — ^ obtain that 
t<k + l. Since l-l,-k>0 and {v{Ei)^) > -2, we get that 

>fcM^-(/-i-fc)fc. 
- 2/ ^ ' 

If r > 1 or > 1 for some j, then for a general filtration, there are Ei and Ej such that Ext^(£^j, Ei) ~ 0. 
Therefore we get that ^j^^- dimExt^(£'j, i?i) < (fc + l)fc/2 — 1 for a general filtration. Then the moduli 
number of these filtrations is bounded by 



{^^)-k^^ + i^-^-m+'-^^^-^ + t<{{v^) + l)-[k'-^-lk 



^ \ ' - 1 + t < {{v^) + 1) - ( - Zfc + -— : ^ + 1 

<((«^> + l)-( + 1 



. 2? 

Therefore we get a desired estimate for this case. Li particular, our lemma holds for the case where x\v j i(v) > 
1. For the case where k = 1 for all i and r — 1, see [Y5, sect. 5.3]. □ 



1.3. Estimate on properly semi-stable sheaves. 



Lemma 1.6. (1) Let E he a stable sheaf and F a semi-stable sheaf such that v(F)/ ikF = v{E)/ rk£'. 
Then Hom(i?, F) ® E ^ F is injective. In particular dimHoni(£', F) < rkF/ rkE. 
(2) Let E be a fi-stable sheaf and F a ^-semi-stable sheaf such that ci{F)/ v'kF = ci{E)/TkE. Then 
Hom(i?, F) iS) E F is injective. In particular diniHoni(£', F) < rkF/ rkE. 

Lemma 1.7. Let v be a Mukai vector of (w^) > (we don't assume the primitivity of v). We set 

M{vY'"' := {E e M{vy''\E is properly semi-stable }. (1.8) 

Then dmiMivY"' < (v^) . In particular, if MivY" ^ %, then M{vY ^ and dimAl(w)^" = (w^) + 1. 

Proof. We set v — Iv' , where v' is a primitive Mukai vector. We shall prove this lemma by induction on /. 
Let El be a stable sheaf of v{Ei) = liv' and E2 a semi-stable sheaf of v{E2) — hv' , where h -\- h ~ I- By 
induction hypothesis, dim Ai'^'' {vi) = (vf) + 1, i = 1,2. We shall estimate the dimension of the substack 
J{vi,V2) whose element E fits in an extension 

^ El ^ E ^ E2 0. (1.9) 



By Lemma 1.6, dim Ext (£'2,i?i) — dimHom(£^i, £^2) < h/h- Moreover if £'1 is general, then Hom(£^i, £^2) = 
0. Hence by Lemma 5.1 in Appendix, we get 

dimJ(t;i,U2) < dimA^(ui)'"' + dim A^(?72)'"* + (wi,W2) +max{l2//i - 1,0} 

= (^'i) + («2> + 2+ (ui,W2) +max{/2//i - 1,0} (1.10) 

= +1)- ((t;i,«2) -max{?2//i,l}). 

Since 

{vi,V2)^li— + l2^>l, (1.11) 
we get dim J(t;i,U2) < W^)- Therefore we get our lemma. □ 
1.4. Semi-stable sheaves of isotropic Mukai vector. 

Lemma 1.8. Let w be a primitive isotropic Mukai vector o/rkw > 0. Then dim Al(Zii;)'"* — I. 

Proof. Let be a semi-stable sheaf of v{E) — Iw. We shall first prove that there are stable sheaves 
El, E2, . . . , Ek of v{Ei) = w such that 

^; = ®tl^^^, (1-12) 

where Fi are S'-equivalent to E®^^ . 

Proof of the claim: By the proof of Mukai [Mu2, Prop. 4.4] (Fourier-Mukai transform for if*(X, Q)), there 
is an element i?i oiMniw) such that Hom(£'i, i?) 7^ 0. By induction hypothesis, E/Ei = Fi ® F2 © • • • ©Ffc, 
where Fi is S'-equivalent to "-i > and Fi, i > 1 are S-equivalent to Ef"'\ Since Ext\£',,£;i) = 

for i > 1, Ext^(®i>i£i, El) = 0. Therefore (BiyiFi is a direct summand of E, which implies our claim. 
Let El be a stable sheaf of v{Ei) = w. We set 

J{l,Ei) := {E e M{lwY'\E is S'-equivalent to Ef}. (1.13) 

This is a closed substack of Ai{lwY'' (see Appendix 4.3). We shall next prove that 

dimj^(/,£;i) < -1. (1.14) 



Since Ei is parametrized by the surface Mh{w), (1.14) implies that 

dimM{lwY"^l. (1.15) 



For more details, see Appendix 4.3. Proof of (1.14): We set 

J{l,Ei,n) := {E G J{1, Ei)\AunYiom{Ei, E) = n}. (1.16) 

By upper semi-continuity of cohomologies, this is a locally closed substack of J{l,Ei). li n = I, then 
Jll, El, I) = {Ef^} and it is a closed substack of J{1, Ei). For an element E of J{1, Ei,n), there is an exact 
sequence 

Q^nou\{Ei,E)®Ei^ E ^ E' ->Q (1.17) 



where E' e J^{l — n, Ei, n'). The moduh number of E which fits in this type of extension is equal to dm\J^{l — 
n,Ei,n') +nTi' -n^. Indeed, Iiom{Ei, E) Ei{= ijf") belongs to J{n,Ei,n) and A\mJ{n,Ei,n) = --n? . 
Hence by the proof of Lemma 5J , we get the equality. Therefore we see that 

Y,n,n,+A < -1. (1.18) 

i=l / 



dimj'(/, £'i) = — min 

l=ni+n2-\ h«s 

ni ,712 ,.. .,ns >1 




□ 



Remark 1.1. If Mh{w) has a universal family, then Fourier-Mukai transform is defined [Br]. Then Ai{lw)'"' 
is transformed to the stack of 0-dimensional sheaves on Mh(w). In this case, by using [Yl, Thm. 0.4], we 
can get our lemma. 



1.5. Lemma [).4 and its extensions. 



1.5.1. Proof of Lemma 0.4- Proof of (1): By our assumptions, we have 

ri 



degEi ^ degifijEi) ^ degE 



rkEi " rkip{Ei) " rk£; r' ^^'"^^^ 

By Lemma ^T^ , degLp{Ei) / Tkip{Ei) = d/r. Hence f{Ei) coincides with E except finite points of X. Thus 
if is surjective in codimension 1. We shall next prove that kertp is a /^-stable vector bundle. Since ri — Ir 
and di — Id are relatively prime, we shall prove that ker ip is /z-semi-stable. If ker ip is not /i-semi-stable, then 
there is a subsheaf / of kertp such that / is ^-stable and di/n > deg(J)/ rk(/) > (rfi — ld)/{ri — Ir). Since 
d/r > di/ri, we see that 



1 



> 



deg(J) 



> 



1 



r{ri — Ir) r rk(/) rrk(/) 
Ir, which is a contradiction. Therefore kenp is a /i-stable vector bundle. 



(1.20) 



Hence rk(/) > ri 

Proof of (2): If cj) is not injective, then {di 
d/r > di/ri, we see that 



ld)/iri-lr) < deg{(j)iE'))/Tk{(j)iE')) < di/n. Since 



1 



d 

> 

r 



deg(0(£;')) 



> 



1 



(1.21) 



r{ri-lr) r Tk{(j){E')) ' rTk{(t){E'))' 

Hence rk{<j){E')) > ri — Ir, which is a contradiction. Thus (p is injective. Assume that E is not //-semi-stable. 
Then there is a /Lt-stable quotient sheaf F oi E such that degF/ rkF < degE/ rkE. Since F is also a 
quotient sheaf of Ei , we have 

di degEi degF degE d 
ri 



By Lemma 1.3, we get ikF > ri 



vkEi i-kF rkE r' 
which is a contradiction. Therefore E is /i-semi-stable. 



(1.22) 



□ 



Remark 1.2. In order to define deg(£') in Lemma y\A\ we assumed that NS(X) = ZH. However deg{E) is 
still defined if {ci{E), H)\{H, D) for all D e NS(X). Hence Lemma D.4 also holds under this assumption. 

1.5.2. Extensions of Lemma O.4 and [Y5, Lem. 4-1]- We shall extend Lemma and [Y5, Lem. 4.1]. Let 
{X, H) be a polarized smooth projective surface of NS(X) = ZiJ. 



Lemma 1.9. Let E he a fi-semi-stable vector bundle of Tk(E) 
non-trivial extension 

^ Fi ^ E ^ F2 ^ 0, 



Ir and deg{E) — Id which is defined by a 



(1.23) 



where Fi and F2 are fj,-stable vector bundles 0/ deg(i^i)/ rk(i^i) — deg(i^2)/ i'k(i^2) = d/r. Let Ei be a 
fj,-stable vector bundle in Lemma O.4 . Let ip : Ei E be a non-trivial homomorphism. Then ip is surjective 
in codimension 1, or (p{Ei) is a subsheaf of Fi. In particular, i/ Hom(£^i, Fi) — 0, then ker (p is ^-stable. 



Proof. We assume that tp is not surjective in codimension 1. By Lemma 1.3, deg((/3(i?i))/ rk((^(£'i)) = d/r. 
Assume that >p>{Ei) — > F2 is not 0. Then the //-stability of F2 implies that it is surjective in codimension 1. 
By the /i-stability of Fi, F iOip jEi) — 0. Thus we can regard ip{Ei) as a subsheaf of F2 . Let e G Ext^ (F2 , Fi ) 
be the extension class of ( 1.23 ). By the homomorphism Ext^(i^2, ^1) Ext^((^(i?i), Fi), e goes to 0. Since 
Fi is a vector bundle, Ex?(F2/<^(£^i), Fi) = 0- Hence Ext\f2,i^i) ^ Ext^ (i^ (£;!), Fi) is injective. Thus 
we get that e = 0, which is a contradiction. Hence ip{Ei) ^ F2 is a 0-map, which means that (p{Ei) C Fi. 



The last assertion follows from the proof of Lemma 0.4 



□ 



Lemma 1.10. Keep the notations in Lemma l.i. Assume that Exi^{Ei,F2) = 0. Then a non-trivial 
extension of E by Ei is fi-stable. 
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Proof. Let E' be a non-trivial extension of E by Ei. 

^ El ^ E' E 0. (1.24) 
We shall prove that E' is //-stable. We consider the following diagram which is induced by the extension 





F. 



E, 



E' 



E 



(1.25) 



El 



E" 



Fi 









By Serre dua lity, Ext"^ (f 2 , Ei ) = 0. Hence ExtVs, Ei) Ext^(Fi, i?i) is injective. Thus the last horizontal 
sequence of (1.25) does not split. By Lemma 1.4, E" is //-stable. If the middle vertical sequence splits, then 
E ^ Fi (B F2, which is a contradiction. Hence E' is a non-trivial extension of F2 by a locally free sheaf E" . 



By the construction of E" , the conditions in Lemma 1.4 hold. Hence applying Lemma 1.4 again, we see that 
E' is /t-stable. □ 

1.5.3. Extension of [Y5, Lem. 4-2]- Let {X,H) be a polarized K3 surface of Pic(X) = ZH. Let Ei be an 
exceptional vector bundle of v{Ei) := ri + diH + aiuj and let v — l{r + dH) + ato be a primitive Mukai 
vector of dri — dir = 1. We set 

Mjj(w)f := {E £ MH{v)^''\dimlioin(Ei,E) = i - {v,v{Ei))}. (1.26) 

Assume that ri > Ir. Then we get the following estimate which is necessary for the condition section 0.2 
(a). 

Lemma 1.11. (1) If {v,v{Ei)) < 0, then 

codimM«(«)M. MH(w)f > -{v,v{Ei)) + 1 > 2 

for i > 1. 

(2) If{v,v{Ei)) > 0, then 

codimM„(^).s,o.{MH{v)';' nMH{vr'''n > {v,viEi)) + 1 > 1 
for i > {v,v{Ei)), where Mh(w)^''''°'= = Mh{vY' n Mff(w)'°^ 
Proof. We set 

' E e Mh{u), dimHom(i;i,i;) = i + 1 - (i',i'(£^i)) , 1 
E/Ei is a /i-stable sheaf of v{E/Ei) = v | ' 



(1.27) 



(1.28) 



Nr-^iEidE 



(1.29) 



where u — v + v{Ei). Then we see that dim Ni < i — {v, v{Ei)) +dim Mh{u) = (v, v{Ei)) + dim Mh{v) — 2. 
Let n'^ : Ni MH{vy be the morphism sending [Ei d E) e to E/Ei e Muivy ■ By Lemma [l^, 
Tr'^{Ni) = Mh{v)1^ and tt'^~^{E/Ei) is isomorphic to the projective space P(Ext^(i?/£'i, i?i)^). Hence we 
get that dimMff(f;)f'' = dimA^, - (i - 1) < dimMjj(v) + {v,v{Ei)) - 1. Thus (1) holds. 

We next prove the second claim. Hence we assume that i > {v, v{Ei)). For E E A/// (u)f'* n Mh{vY^'^°'^ ^ 
we choose a homomorphism (j) : Ei ^ E. By Lemma 0.4, is surjective in codimension 1. We set 



G := coker(£;^ ^ E':^). By Lemma bj, F is a stable sheaf of v{G) = 



v{Ei 



It is easy to see 



that dimHom(i?]^, G) = dimExt^(i?i, i?) + 1 — {v, v{Ei)) + i + 1. Hence <j) : Ei E is parametrized by an 
open subscheme of a projective bundle of dimension {{v, v{Ei)) + i) over the subscheme MH{w)i, where 

MHiw)i : = {G e MHiw)\diiRRom{E^ ,G) = {v,v{Ei)) + i + 1} 

= {Ge MH{w)\dimEom{E'^,G) = {w,v{Eiy) + (i-l)}. 

Thus we see that 

dimMniv)^' n Mnivy''-^"^ < (w^) +2+ {{v, v{Ei)) + i) ^ {i - I) 

= {v^)+2-{{v,v{Ei)) + l). 



(1.30) 



(1.31) 



□ 

2. Case A 

2.1. Estimate. In this section, we fix a primitive Mukai vector r + S NS(X). We assume that 

((e2) + 2)/2r^Z. (2.1) 



We shall prove Theorem D.l for a primitive Mukai vector v := l{r + + auj G H*{X,'L). We shall first 
estimate the dimensions of various locally closed substacks of M (u) . 

Lemma 2.1. If M{vy ^ 0, then (v^) > 0. // the equality holds, then Miv^'' = M{vy' . 

Proof. Let i? be a /x-semi-stable sheaf of v{E) = v and E is 5-equivalent to (Bf^iEi with respect to /x-stability, 
where Ei,l < i < s are /^-stable sheaves. We set 

v{E,) k{r + C) +a,uj,l <i < s. (2.2) 
By our assumption (|1]), (w(£',)2) = h{h{^'^) - 2ra^) ^-2. Thus {v{E,)'^) > for aU i. Since 

^.±^. ,2.3) 

we get (v^) > 0. If = 0, then {v{E,f) = for all i. Since {v{E,f) / Tk{Ei)^ = (f^) _ 2a,/W, and 
x{Et)/Tk{Ei) = 1 + fli/r^i, we see that x{Ei) / Tk{E.,) = x(-E)/rk(£') for aU i. Thus £; is semi-stable. □ 

Corollary 2.2. If {v^) — 0, then Ai{v)^^'^^ consists of locally free sheaves. 

Definition 2.1. Let w — lo{r + £,) + qquj be the primitive Mukai vector such that (w^) = 0. 



By Lemma 2.1 and Corollary Mh{w) consists of /i-stable locally free sheaves. 
Lemma 2.3. (1) 

dim(Al(w)^"- \M{vD < {v^). (2.4) 

(2) Assume that {v^) > 0. Then 

dimiMiv)"'' \M{vy) < (v^). (2.5) 
In particular, if M{vy ^ 0, then M{vY ^ and dimA^(w)^''^ (u^) + l. 



Proof. By Lemma 1.7, it is sufhcient to prove (1). Let be a /i- semi- stable sheaf of v[F) — v. We assume 
that F is not stable. Let 

C i^i C F2 C • • • C i^. = F (2.6) 
be the Harder-Narasimhan filtration of F. We set 

v{F,/F,^i) = k{r + ^) + a,Lo, l<i<s. (2.7) 

Since x{F^/F,^l)/rk{F,/F,^l) > x{F^+l / F,) / vk{F,+i / F,) , we get that 

an ai a? a, , . 

/>-i>-^>.-.>-^. (2.8) 

Let !F^^ {vi,V2, ■ ■ ■ ,Vs) be the substack of A4(w)^''* whose element E has the Harder-Narasimhan filtration 
of the above type. We shall prove that dim!F^^ {vi,V2, ■ ■ ■ ,Vs) < (w^). Since llom{Fi/Fi^i, Fj/Fj-i) = 
for i < j, Lemma 5^ in Appendix implies that 

s 

dim .F^^ {vuV2,---,v,)^J2 dim M{v,r+J2 ' • (2-9) 

2—1 i<j 



For J < j, by using Lemma 2.1 and (2.8), we see that 

{vi,Vj) = kljiS.^) - {kaj +ljai)r 



lilj{£^'^) — 2ljair + {oilj — ajli)r 



(2.10) 

> {ailj — ajli)r > r > 2, 



where the inequality r > 2 comes from our assumption (2.1). Hence if (vf) > for all i, then, by using 
Lemma 1.7, we see that 

dh-nT"^iv,,V2,...,v,)<i{v^) + l)- \ y,{v^,v,)-s + l\ <{v^). (2.11) 



Assume that {vf) — 0, i.e. Vi — I'^w, li G Z. Then i — 1 and {ailj — ajli) is divisible by l'^. Hence 

{vi,Vj) -l[ = l'i{{w,Vj) - 1) 

> l[ir-l) > 0. 



(2.12) 



In this case, by using Lemma 1.8, we see that 



dimJ^''^{vuV2,...,Vs)<{{v') + l)- \^^{v,,v,)-{l[ + s^l) + lj < («2>. (2.13) 

Hence we get our lemma. □ 

Proposition 2.4. Assume that (u^) > 0. Then 

dim(A^(w)^ XMiv)"') < (v^). (2.14) 
In particular, MivY'" 7^ 0, ij M{vY ^H). 

Proof. Let i? be a stable sheaf and Ei be a /i-stable subsheaf of E such that E/Ei is torsion free. We set 

Vi :— v{Ei) — li{r + S)+ o-i^, 
V2:--v{E/Ei)^l2{r + + a2UJ. ^'^'^^^ 



Since x(^^i)/rk£;i < x{E)/rkE, we get {v{Eif) > and 

Ol 02 

Let J{vi,V2) be the substack of M.{vY consisting of E which has a subsheaf Fi C E. By using Lemma 



^ < ^. (2.16) 



5.1 in Appendix, we shall estimate di'mJ{vi,V2)- By Lemma 1.6, dimHom(_Ei, _B/_Ei) < ^2/^1: f^nd if Ei is 
general, then Hom(i?i, E/Ei) — 0. We shall bound the dimension of the substack 

JV{vuV2) {{Ei,E2) e Mivi^'' x A^(t;2)'^"| dimHom(£;i, ^2) ^ 0}. (2.17) 

For a fixed E2 £ M{v2y , 

#{Er\Ei e Mivi)^',Roin{Ei,E2) ^ 0} < 00. (2.18) 

Hence, by using [Yl, Thm. 0.4], we see that 

dim{£:i e 7W(i;i)''"|Hom(£;i,£;2) ^ 0} < dim Al(t)i)^'' - 2 - (rkwi - 1). (2.19) 

Thus dimA/'(wi, W2) < dim A^(i;i)^'"* + dim 7Vl(w2)^'"* — 3. Moreover, taking ( 1.12| ) into account, if li ^ Iq 
and V2 = l'2W,l'2 & then we get N'{vi,V2) = 0. 



If (ti|) > 0, then Lemma 2.3 implies that dim A^(w2)^'"' = + 1. Hence Lemma implies that 



dim A^(w)^ - dim J(wi, U2) = niin ( {vi,V2) - ^ + 2, {vi,V2) - 1 



/.^ + ^2^-max(^-2,l|>0 



We next treat the case where = 0. Then V2 = Vr^w^V.^ G Z. By Lemma 2.3 (1) and Lemma 1.8, 
dim A^(w2)''^'' = + 1^- If h = ^o, then ;2/'i = ^2- So we see that 

dimA^(u)'* - dim J(t;i,i;2) = min (/2((wi, w) - 1 - 1) + 3,;2((wi, w) - 1)) 

= min{;2((ao^i - ai'o)?' - 2) + 3, l2{{aoli - ailo)r - 1)} > 0. ^2.21) 

□ 



Remark 2.1. By the proof of Lemma |2.3| and Proposition 2.4, we see that 

codimMivy'^^{M{v)i'''\M{v)'''') > 2 for r > 3. (2.22) 

Moreover if r = 2, then the general member E of A4{v)'' \ A4{v)'^'^ fits in the following exact sequence 

0->Ei-^E^E2^0 (2.23) 

where Ei is a ^-stable vector bundle and E2 is a /^-stable vector bundle of v{E2) = w. 

Indeed, if codim^(„)= (7W(u)* \ M{v)^'') = 1, then (1) (v^) > 0, = /2 = (w?)/2 = (vj)/2 = 1, or (2) 



r = 2, ^2 1 and aoh — ailo = 1. By (2.16), case (1) does not occur 



2.2. Proof of Theorem p.lj for case A. 
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2.2.1. The case of r > 2. In the same way as in the proof of Theorem |l.l| , we shaU prove Theorem |0.l| if 
r > 2. Let v = l{r + + aui he a primitive Mukai vector on a K3 surface X such that r + ^, ^ e NS(X) 
is primitive. We claim that we can find a primitive Mukai vector v' = l{r + d! H') + a'u) on a polarized K3 
surface {X',H') of Pic(X') = ZH' and an exceptional vector bundle G of v{G) :— vi = ri + diH' + aiu)' 



such that (1) r + d' H' is primitive, (2) {v^) = ((u') ), (3) a 



mod /, (4) d'ri - dir = 1, (5) n - Ir > 2 



and (6) {v{G), v') — —1, where lo' is the fundamental class of X'. 

Proof of the claim: We choose integers ri, di and d' such that ari = 1 mod / and {ri,r) = 1. We can easily 
choose such an integer ri oi ri — Ir > 2. Then we can choose d' and di of d'ri — dir = 1. Let q be an integer 
of an +ql = 1. We set fc(s) := ri{qr + ris) - r^s = (C^)/2 G Z. Then k{s) = {ri{v'^)/2l + r)ri/l - r^. 
Since (ti^) > 0, fc(s) > rr^// - > 0. Let (X',H') be a polarized K3 surface such that Pic(X') = ZH' and 



(F'^) = 2k{s). We set 



Then we see that 



Ir- 



Id'H' + {/((I + d'ri)dis + d'^qn - rd'^) + a}uj' , 
' diH' + {ni-d'^ + djs) + djrq + 2d'}uj'. 




Vl,V 



= 2l{ls-ra) = 
'\ = -1. 



(2.24) 



(2.25) 



1.1 



Since ri and c?i are relatively prime, Theorem 
v{G) — vi. Then G and v' satisfy our claims. 

We shall consider reflection defined by G. We note that 

w := -Ra,^{v'Y = (ri - Ir) - (di 

Since ri 

Mh'(w) 7^ and Theorem D.l holds for this space. Since ri 



implies that there is an exceptional vector bundle G of 



ld')H' + (fli — a')uj'. 
r and di — Id' are relatively prime. Theorem [l.l| for a Mukai vector w of £{w) 



(2.26) 



1 implies that 
Ir > 2 and Mh'{w) consists of /i-stable 
sheaves, [Yl, Thm. 0.4] implies that there is a /x-stable vector bundle £' of v{E) — w. By (6), we see that 



— 1. Hence there is a non-trivial homomorphism : ^ E'. B\ 



(2) , we see that coker(0^) is a /i-semi-stable she af o f «(cokcr(0^ )) = v' . Then Lemma 
2.4 imply that Mh'{v')'^^ ^ 0. Applying Lemma |o!^ and Lemma 1.11 , we get a birational map 

Mh'{v') >Mh.{w) 



using Lemma p.4 
and Proposition 



(2.27) 



sending F G Mu'W)'^" n Mh'WY"" to coker(i^^ G^), which means that Theorem ].l (1) and (2-1) hold 



for Mh'{v'). Then Proposition 1.2 implies that Theorem p.l| (1) and (2-1) also hold for Mh{v), where H is 
a general ample divisor on X. 

Moreover by Remark |t|, we can naturally identify H'^{Mh{v'),Z) with {M h (w) , Z) . Let J^" : 
H*{X,Z) — > H*{X,'Z) be an isomerty of Mukai lattice defined by {x) — Ry^{xY . Then it is easy 
to see that the following diagram is commutative (see the computation in [Y5, 2.4]). 



Hence (2-2) also holds if r > 2. 



□ 



2.2.2. The case ofr — 2. We next treat the case where r = 2 . It is sufhcient to extend the birational map 
( ^.27|) to a general member E £ Mh '{v') which fits in ( 2.25 ). Let G be the exceptional vector bundle in 
2.2.1. By using Lemma 1.9 and 1.10 , we can prove t he fo llowing: Assume that {v',v{G)) = —1. Then, for a 
general member E which fits in the exact sequence ( 2.23 ), 

(1) Ext\G,E) = 0, 

(2) (j) : Hom(G, E) (g) G ^ E is surjective in codimension 1 and ker (j> is stable. 



xw — yuj, where x,y £ Q and x,y > 0. Since {v',v{G)) = 
Hence {v{Ei),v{G)) > and {v{E2),v{G)) < 0. Applying 



Proof. Since {{v'Y) > 0, we can write v' - 
x{w,viG)) +yrk(G) = -1, {w,viG)) < 0. 

Lemma |l.ll| , we may assume that Hom( G, Ei) — Ext^(G, i?2) = 0. By Lemma 
argument as in the proof of Lemma |l.ll| (1). Hence (1) holds. Applying Lemma 

Therefore the dual of 4> ■ G^ is injective and the cokernel is stable. Thus the reflection i nduc es a 

birational map Mh{v) \ Z — > Mh{v{GY — v^) such that codimj\/^(„) Z > 2. Therefore Theorem OT also 
holds for this case. □ 



1.10 , we can use the same 
Lg we get (2). □ 



Remark 2.2. Under the assumptions for case A, the following holds. 
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(1) Mh{v) 7^ if and only if (i;^) > 0. 

(2) MhIvY'' 7^ if and only if {v^) > 0. 

In particular, Mh{v) contains /i-stable vector bundles if Mh{v) ^ 0. 

3. Case B 

In this section, we assume that there is a (—2) vector wq of the form vq = r + ^ + buj, b E Z. Let Eq be the 
element of Mh{vo). We shall prove Theorem 3.1 for a primitive Mukai vector v := Ivq — aus. If (u^) > 2P, 
then the same proof in section 2.2.1 works, because of Lemma 1.5. Hence we may assume that (w^) < 2P. 

3.1. The case of (w^) < 2P. We first treat the case where (w^) < 2P. Clearly (w^) > -2, if Mh{v) ^ 0. 
If (w^) = —2, then 2l{aikvQ ~ I) — —2. Hence we get I — 1. This case is covered in Theorem So we 
assume that (w^) > 0, that is 

Z < ark'yo < 2/. (3.1) 



(3.2) 



Based on the next key lemma, we shall prove Theorem 0.1 in 3.1.2 and 3.1.3. 
Lemma 3.1. codim^./^i-^,) (MH(f ) \ Mniv)''"'^) > 2 unless 

\{Tkvo)vQ - uj, 
\Ivq ~ {I + rkvo = 1. 

3.1.1. Proof of Lemma We set 

M{v,ny -.^ {E e M{vy\dim{E'''' /E) ^ n}. (3.3) 

For E e M{v,ny, let 

C i^i C F2 C • • • C = £;'''^ (3.4) 
be the Harder-Narasimhan filtration of i?^^. We set 

Vi := v{Fi/Fi^i) = kvo - OiLO, I <i < s. (3.5) 
We shall estimate the codimension of the substack 

M{v, n;vi,V2,..., Vs) := {E e M{v, n)^|£;^^ € T"'^{vuV2, . . . , i-s)}, (3.6) 



where (wi,W2, ...,«=,) is defined as in the proof of Lemma 2.3. We divide our consideration into two 
cases 

(I) s > 2, that is, E'^'^ is not semi-stable. 
(II) s = 1, that is, E'^'^ is semi-stable. 
Case (I). (I-a) If {v\) > 0, then we can use almost the same arguments as in the proof of Lemma ^.3| . The 
difference comes from the inequality r > 2 which was used in (2.10) and (2.12). Thus 

dimJ^"^ivi,V2....,v,) < («(ii;^^)2> + 1. (3.7) 
The equality holds only if r = 1 and {vf) = 0. Since Fs is locally free, Fi is also locally free. On the other 



2^ (3-8) 



hand, if r = 1 and (vl) = 0, then Al(wi)*'' consists of non-locally free sheaves (cf. Lemma 1.8). Therefore 
the equality does not hold. Hence, by using [Yl, Thm. 0.4], we see that 

dim A^(w, n;vi,V2, ■ ■ ■ ,Vs) < ((w — nuj)'^) -t- 1 -I- n(rk v + 1) 

< (v^) + 1 - n(rk'i; - 1). 

Thus we get a desired estimate 

codim^(^,)= M{v,n;vi,V2, . . .,Vs) >2. (3.9) 

(Lb) We assume that (vf) < 0. Then Fi = E®^\ which implies that dimMiviY" = -if = (vl) + if. For 
convenience sake, we set 

s 

v'2 '■— Vi — l'2Vo — a'2Uj. (3.10) 

1=2 

(I-b-1) We first assume that 

{{v',y)=2l'^{a',rkvo-l'2)>0. (3.11) 
Then, since (vf) > for all i > 2, dim J^^^(w2, . • . , w^) < {{v'2)^) + 1- Thus we get 

(«(i?^^)2) + 1 - dim.F^^(«i,i;2, ■■■,v,)> {vuv'2) - ll (3.12) 

We shall prove that 

{vi,V2) -ll + n{likvo-l)>2. (3.13) 
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Proof. Since E is stable, Fi O E satisfies that 

x{Fi n E) xim 

rkFi TkE' 

Since xi^i H i?) > x(^i) ~ ^i^d ui = ^iWq (i-e. ai = 0), we see that 

71^2 — 02^1 > 0. 



By our assumption ( |3.lD , (aj + n) rkwo < 2(^i + Zj)- I3y using (3.11), we see that 

2li + l'2-l 



n < 



vkvo 



By using ( ^.11 ) and ( [3.15 ), we see that 



nil rkwo > ^1 + 1. 



(3.14) 
(3.15) 
(3.16) 
(3.17) 



Assume that rkvQ > 2. Then (3.16) implies that n < + 12/2. By using (3.11) and (3.17), we see that 

{vi ,V2) - if + n{l rkvo-1) =li{-2l'.2 + 4 rk wq) + n{{li + l'2)rkvo - I) - ll 

— — + ^1(12 rkwQ — ^2) + '^'1 rkwo + ?i?2 + l'2{n'i:kvQ ~ li) ~ n — l\ 
>Zi(a2rkvo — I2) + l'2{m:kvo — li) + \ — n ^g) 
>Zi + ^2 + 1 - > 1- 

Assume that rkug = 1. Then similar computations work if a2rkwo — ^2 > 1- we assume that rkug = 
a'2 rkuo — ^2 = 1- Then (3.17) implies that n — > 0. Hence we see that 



/i(-2;^ + 4rki;o) +7i((Zi +Z2)rkt;o - 1) - = (^1+^2 - l)(?^-^i) > (^ - 1) > 1- 



(3.19) 



If the equality holds, then I — 2 and n — li = 1. In this case, we get that li — I2 — \ and 71 — 2. By (3.15), 
we get a contradiction. Thus the left hand side of ( |3.19 ) is greater than or equal to 2. Therefore (3.13) 
holds. □ 



By §11) and [Yl, Thm. 0.4], we get a desired estimate 

codim^(^,). M{v, n\vi,V2, ■ ■ ■ , Vs) > 2. 

(I-b-2) We next assume that 

((«^)2) =2/^(4 rki,o-?i)=0. 



(3.20) 
(3.21) 



Then s = 2 and (wc^i) = -'2- Since Hom(i;o, F2/F1) = 0, dimHom(F2/i^i, So) > Z^. By Lemma |l.6|, 
coevaluation map 

■ F2/F1 -^Eo® llom{F2/FuEny (3.22) 
is surjective in codimension 1 (cf . Lemma |l.6| ) . Therefore we get 

(i) dimHom(F2/i^i,So) =^2, 

(ii) dimExt^(i^2/-Fi,£^o) = and 

(iii) tp is isomorphic in codimension 1. 

By the definition of Harder-Narasimhan filtration, ip is not isomorphic. Thus F2/F1 is not locally free. By 
(ii), -E^^ = Fi ® F2/F1, which contradicts the locally freeness of E'^^. Thus this case does not occur. By 
(Ta), (Tb-1) and (I-b-2), we get a desired bound for the case (I). 

Case (II). We divide our consideration into three cases (Il-a) (u^) > 0, (Il-b) (u^) — and (II-c) (w^) < 0. 

(n-a) If {vf) > 0, then 



dim A^(w, n; vi) = dim A1(ui) + ri(rkv + 1) 
= (vl) + 1 + n{rkv + 1) 
= (w^) + 1 - n{rkv - 1). 



(3.23) 



Hence if rkw > 3, then codim^vij^j^s M(v, n; vi) > 2. If rkw = 2, then the condition (3.1) implies that a = 3. 
Hence we get (vf) — {v^) — 2nikv < 0. Therefore this case does not occur. 

(Il-b) If {vf) — 0, then the argument in (I-b-2) implies that M{viY'' consists of non-locally free sheaves, 
which is a contradiction. 



(H-c) We assume that (vl) < 0, that is, = Ef. Then (|3TD imphes that nrkwo - ^ > 0. 
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(II-c-1) We first assume that nTkvQ — I > 1. Then we get 



codim^(j,)s A4{v, n) — {2nl rkvQ — 2P + 1) — {n{l rkwo + 1) — 



= n{l ikvo - 1) - - 1) 



> 



-{likvo 
rk vo 

(; + l)(rkt;o- 1) 
rk Vo 



1) - {l^ 
>0, 



1) 



(3.24) 



and the equality holds if and only if rkvo ~ 1 and nikvo — I = 1- By the computation of (3.24), it is easy 
to show that codim^vij-^-js A4{v,n) > 2, if rkvo ^ 2, or nrkwo — ^ > 2. 

(II-c-2) If nrkvo ^ ^ = Oi then the primitivity of v implies that n = 1. Therefore we get a desired bound 
for the case (II). 



By (I) and (II), we complete the proof of Lemma 3.1 



□ 



3.1.2. Components containing locally free sheaves. We shall prove Theorem 0.1 unless v = (rkwo)wo ~ or 
rkuo = 1 and v — Ivq — {I + 1)uj. For a locally free sheaf E G A4{vy, we consider the dual of E. Since 
{vo,v) — arkwo ~ 21 < and E is stable, /' :— dimExt^(£'o, i?) > 2/ — arkug > 0. By Serre duality, we get 
an exact sequence 



0~^{E- 



E^ 



where F £ M{v'^ 



I'v^)'"'". Then we see that 



Since I' < L we see that 



Hom(^;^,F) = Ext2(^;^,F) = 0. 



(w(F)2) = 2l{arkvo - I) ~ 2l'{l' - 21 + arki;o) 



^ 2{l - l'){aikvQ - I + I') 
> 2{l-l')l > 2(1 -I'f. 



Hence, by Lemma L5, we get dimA^(u^ — )^^* — {{v — I'vp f') + 1. Taking into account 
that the moduli number of E^ which fits in the exact sequence (3.25) is given by 



dimX(w^ - I'v^y +Aiu,Gr{{v'' - l'v^,v^),l') = (v^) + 1 - I' {I' + {vo,v)), 
(cf. [Y5, Lem. 2.6]). We set 

Ext^(£;o,£;) 



Mniv)* 



E € Mh{v) 



loc 



0, 

coker(Hom(i;, Eq) ® E^ ^ E"") e Mh{Rvo{v^)) 



(3.25) 
(3.26) 

(3.27) 

we see 
(3.28) 

(3.29) 



Then, by using Lemma 1.5, Lemma 3.1 and ( 3.28| ), we see that codimAf^(^) (M// (u) \ Mh{v)*) > 2. Since 
Theorem 0.1 holds for Mj:/(i?„Q (u^)), (_/?„„ (u^)) is irreducible. Therefore the morphism Mh{v)* — > 
MniRvoiv"^)) is birational, if Mh{v) ^ 0. Conversely, for a /i-stable vector bundle F G (i?.„„ (w^)), we 
consider the universal extension 



^E- 



E' ^ F ^0. 



(3.30) 



V. This means that Mh{v) ^ and Theorem |0.l| (1) 



^ Yjyii^{F,E'^y 

We claim that {E'Y is a stable sheaf of v{{E'Y) 
(2-1) hold for this case. The proof of Theorem 0.1 (2-2) is similar to the proof for case A. 
Proof of the claim: Assume that [E'Y is not stable. Since {E'Y is a /i-semi-stable vector bundle, there is 
a subbundle G such that 

(1) {E'Y /G is torsion free, 

(2) v{G) = IiVq — aiuj and 

(3) ai/h < a/l. 

Since F^ is /x-stable and {E'Y /G is torsion free, F'^f^G = 0, or F^. If F^nG = 0, then G ^ Ext^F, Eq)®Eo 



is injective. Since the slopes of G and Eq are the same and G is locally free, we see that G = E® ^ , which 



means that (3.30) is not the universal extension. Therefore nG must be equal to F^. This means that G 
contains F^ and v{G/F'^) — {li+l — arkwo)wo — (oi — a)uj. Since G/F^ is a subsheaf of Ext^(F, Eq) (g) Fq, 
ai — a > 0. On the other hand, (3) and li < I implies that ai < a, which is a contradiction. Hence {E'Y is 
stable. □ 
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3.1.3. Non-locally free components. We shall prove Theorem 3.1 for 

_ J (rkwo)vo - t^, 



(3-31) 

Ivq — {I + rkvo — 1- 



Proposition 3.2. Assume that v = (rkuo)wo — Then Mh{v) = X. 

Proof. By the argument in (I-b-2), E e Mniv) satisfies that E'^'' = E^'^'">. Since v = v{E^'^'">) - ui, E 
is the kernel of a quotient E®'^^'"" C^, x €z X. We shall construct a family of stable sheaves {£x}xex of 
v{£x) = V and prove our proposition. For convenience sake, we set Xi := X, i — 1, 2. Let A C X2 x Xi be 
the diagonal. We denote the projections X2 x Xi Xi, i = 1,2 by p^. We shall consider the evaluation 
map 

(P-.E^^Eo^ {E^ K £;o)|A ^ Ca. (3.32) 

Then it is easy to see that (jj^ ■— (f'llxyxXii ^ G ^2 is surjective and the induced homomorphism 

Hom(£;o, {E^ M E„)\{x}xx) ^ Hom(£;o, C,) (3.33) 

is an isomorphism. Hence ker^^, is stable. Since Oa is flat over X2, £ ker0 is flat over X2 and 
£\{x}-x.x = ker(/)a;, X e X2 is stable. Thus we get a morphism X2 ^ Mh{v), which is an isomorphism. □ 

Corollary 3.3. R^^ is the Fourier-Mukai transform defined by £. 
Proof. Let : D(Xi) ^ 0(^3) be the functor defined by 

T{x) := RHomp,(f ,p*(a;)),x e D(Xi) (3.34) 

and J- : T){X2) D(Xi) the functor defined by 

T{y) ■.^B.p^,{£®p*2{y)),y^T>{X2). (3.35) 

Then T\2\ gives the inverse of T. Let be a coherent sheaf on Xi such that 

}ioiJi{Ea,E) = 0, 

2, ^ (3-36) 

¥.y±^{En,E) =Q. 

We shall prove that E satisfies WITi for JF, i.e. 

Homp,(f ,pt(i?)) = Ext2^(£:,pt(i?)) - (3.37) 

and J^^{E) := F,xtp^{£ , pl{E)) fits in the universal extension of E by Eq: Since Oa is flat over Xi and Kx 
is trivial, we get 



nom\OA,pUE)) = ' ^'^\ (3.38) 



By using local-global spectral sequence, (3.36) and (3.38), we see that (3.37) holds and we get an exact 
sequence 

Exti(£;o,£;) ®En^ Ext^^(£,p*(i;)) -^E^Q. (3.39) 
We claim that this sequence gives the universal extension of E hy Eq. 

Proof of the claim: If it is not the universal extension, since Ext"'^(_Eo, -B) = Ext^(£', i?o)^, Eq must be a 
direct summand of J-^(E). Since E satisfies WITi for T^{E) satisfies WITi for which implies that Eq 
also satisfies WITi for T. In particular, R^pi^{£ ® P2{Eq)) = 0. On the other hand, a direct computation 
shows that R^pif{£ ® P2{Eq)) ~ Eq, which is a contradiction. Therefore ( 3.39| ) is the universal extension. 
Thus T^{E) is the reflection of by wq. □ 

Remark 3.1. As in [Y7], we define : H*{Xi,Z) H*{X2,Z) by 

T"{x) :=p2* (ch(£)^ptVtd^P2v/td^K(2;)) e H*{Xi,Z). (3.40) 

Since ch(£:)^ = p*(ch(£;o))>5(ch(i;o)) - ch(OA)'', we also see that T"{x) = -(a; + {x,v{Eq))v{Eo)) = 
~Rv{Eo)i^)- Thus we get the following commutative diagram: 

D(Xi) — ^ D(X2) 

ch ^tdxi I ch ^tdxa (3-41) 

H*{XuZ) > H*{X2,1) 
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Finally we shall treat Mh{Ivo ^ {I + rkvQ — 1. 

Proposition 3.4. Ifikvo = 1, then M h (Ivq ~ {I + 1)lu) = Hilb^^ andOy is an isometry of Hodge structures. 

Proof. We may assume that wo = 1 + oj. Let E be an element of Mh{1 — oj)- We shall first prove that 
^vv ^ Q®i_ ^Yiis^ it is sufficient to prove that 



dim(£;^^/£;) = Z + f . 



(3.42) 



Proof of (l3.42|) : Since = ^ - 1 and £: is stable, Serre duality implies that dim H°{X,E'^) > Hence 
we get an exact sequence 



0->O 



e(/-i) 

X 



(3.43) 



where Iz & Hilb^^^"". If n = 0, then E is locally free. By taking the dual of (|3.43|) , we get a section 
of E, which contradicts the stability of E. We assume that < n < ^ + 1. Then dimExt {Iz,Ox) = 



dim {X , Iz) — l~n. Hence we get a decomposition i?^ 



O 



X 



X 



F^Iz 



Then E'^'^ has a subsheaf Of". The stability of E implies that 



I 



< 



) F, where F fits in an exact sequence 

(3.44) 

(3.45) 



Since xiOT E) > xiOT) 



I, this is impossible. Therefore ( 3.43 ) holds, which implies that 



E'^ 



O 



X 



. tXi^i £ X, it is easy to see that ker0 is 



Conversely for a general quotient 
stable. Thus Mh{1 - w) 7^ 0. 

We shall prove that it is isomorphic to Hilb|^^. For this purpose, we shall consider a functor Q : T>{Xi) 
D(X2)op which is the composition of reflection by v{Ox) with the taking dual functor: 



g{x) :=RHomp,(p^(a:),/A),2^eD(Xi), 



(3.46) 



where we use the same notation as in Proposition 3.2 and D(X2) op is the opposite category of T>{X2)- Then 
Q gives an equivalence of categories. For E E Mh{1 — ^), we shall prove that 

(a) Ext;^(pJ(i;),/A) = 0, i = 0,2 and 

(b) Q'^{E) := Ext^^(pJ(£;), /a) is an ideal sheaf of colength I + 1. 

Then the map MH{l~ijj) — > Hilb^^ sending E to G^{E) gives an isomorphism of moduli spaces. The second 
assertion follow from [Y7, Prop. 2.5] or a direct computation by using the equality 



g{£) = RHom 



i£,OMH{i-u;)xXi) ^Ox2 -'R''Hom{£,OM„{i-Lu)xX2 



(3.47) 



as an element of Grothendieck group of Mh(1 — lli) x X2, where £ is a quasi-universal family on Mh{1^lo) x X . 

Proof of (a), (b): By Serre duality and the stability of E', Ext^(i?, /j^) = for all x E X. Also we see 
that Hom(iJ, J^:) — for x ^ Supp(i?^^/i?). Hence by the base change theorem and its proof, we see that 
(a) holds and Q^{E) is torsion free. It is easy to see that v{Q{E)) = — (-y(£')^) = — 1 + luj. Hence 
v{Q^{E)) = 1 — lu!. Therefore G^{E) is an ideal sheaf of colength I + 1. 

□ 

3.2. The case of (w^) = 2P. We next treat th e ca se where (v^) = 21"^. Let i? be a general member of 
Mh{v) \ Mh{vY^ . Then by the proof of Lemma 1.5, E fits in an exact sequence 



— > E\ — >■ E — > i?o 







(3.48) 



where Ex is a /Lt-stable vector bundle. 

Indeed, if {v^) = 2P, then the primitivity of v implies that (i) v — Ivq — 2uj , I = rkug or (ii) 1; = Ivq — lu, 
21 = rkiiQ. In particular rkvo > I. In the notation of the proof of Lemma L5, we se e th at k — I. Hence E 
fits in the above exact sequence. Then in the same way as in 2.2.2, we get Theorem D.l in this case. □ 



3.3. Some remarks. By the proof of Theorem 3.1 for case B and Lemma 1.5, we also get the following. 

(1) MuivY" 7^ if and only if {v^) > 2P. 

(2) Mff{vy°'^ = if and only if (i) rkw = 1, (ii) v — {rkvQ)vQ — uj, or (iii) rk^o = 1 and v = Ivq — {I + l)uj- 
Combining Propositions 3^, 3^ and Remark 2^, we get Proposition ^T^ . 
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4. Relation to Montonen- Olive duality 



In this section, we shall consider the relation between Theorem 3.1 and Montonen-Olive duality in Physics. 
Roughly speaking, Montonen-Olive duality says that the generating function of Euler characteristics of 
moduli spaces of vector bundles becomes a modular form. In this paper, we concentrate on moduli spaces 
of vector bundles on K3 surfaces. We first describe physical predictions and their modifications. For more 
details and related results, see [MNVW], [V-W] and [G63], [Y4], [Y6]. 

4.1. Physical predictions. We fix a K3 surface X. We regard Z) as a lattice by a bilinear form 

Q{x,y) = — f-^ xy,x,y € H^{X,Z). Let P be a orthogonal decomposition of Z) (g) R as a sum of 

definite signature: 

P:ff2(X,Z)(g)M = Ri^'°®R°'^ (4.1) 

Let Pl{x) = XL, Pr.{x) = xr denote the projections onto the two factors. 

For v = r + £^+aijj€ H*{X,'Z) of ^ S H'^{X,'Z), we choose a suitable complex structure such that ^ 
become holomorphic. Then we define M{v) as a moduli space of stable sheaves on this surface. Let Zr{T, x) 
be ?7(r)-partition function defined in [MNVW, sect. 3]: 

ZriT,x):^ J2 "x(M(t.))"g^g57Q('=iWi)g^;^Q('=iM«)eQ('=iM'^) (4.2) 

rk v—r 

where {t,x) G H x H'^{X,Z) (g) C, g := exp(27r\/^T), e := exp(27rv^) and "x(M(i;))" is a kind of "Euler 
characteristics" of a nice compactification of M{v). 

Remark 4.1. More precisely, Mina han et al. considered Zr{T,0). Combining the computations in [MNVW, 
sect. 6], we propose the definition (|4.2|). 



Unfortunately, there is no mathematical definition of this "Euler characteristics". Since M{v) is smooth 
and compact for primitive v, we can expect that "x(Af(w))" coincides with the ordinary Euler characteristics 
xiMiv)). 

Then Montonen-Olive duality for U (r) gauge group asserts that 
(#) Zr{T,x) transforms like a Jacobi form of holomorphic/anti-holomorphic weight 

{-x{X)/2 + b-{X)/2,b+{X)/2) = (-5/2,3/2) 

(cf. [E-Z]). For a G H^{X,Z), let Z^{t) be F5J7(r)-partition function defined in [V-W]: 

„2> 



Z?ir):= "x(Af(«))"g^. (4.3) 



rk v—r,ci [v)—a 

Then 

Zr{T,x)= Z^{t)Q^At,P,x), (4.4) 

where 

e„,,(T,P,x)= g^'J^^'^g^^f^^^e^f^'-) (4.5) 



is Siegel-Narain theta function (cf. [M-W, Appendix B]). If r — 1, then it is known that Zi{t) — ^y^Tp 
([G61], [V-W]). Hence 



ceH^x,z) j (4.6) 



\iQ{t,P,x) 



r]{T 

where 0(r, P, a;) = X^csff^^^ 'j'''^*'^^''?"'"'^'''^"''^'''"'^'^''- Since Q{t,P,x) transforms like a Jacobi form of 
holomorphic/anti-holomorphic weight (19/2, 3/2), Zi{t, x) transforms like a Jacobi form of holomorphic/anti- 
holomorphic weight (—5/2,3/2): 

Zr('-i,:^ + ^')=(-y^r)-/^(^/^T)3/^e^e^Z,(r,.,-Hx^). (4.7) 



r T r 
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Then Z^ir, x) is given by Hecke transformation of order r of Zi{t, x) ([MNVW]): 

Zr{T,x)^^ J2 dZi "^^^ 



a,fc,d>0 
ad—r 
b<d 



(4.8) 



In particular, Zr{T,x) transforms like a Jacobi form of holomorphic/anti-holomorphic weight (—5/2,3/2) 
and index r. Thus (#) holds. 



Remark 4.2. For PS'C/(r)-partition functions, we get the following: 



a,fc,d>0 
ad—r 
b<d 
a{=ci 



Combining (iA) with the transformation law 

e„,. ( — ,P, — + ^ I = r'"(-\/^r)i9/2(V^r)3/2e^^e^ 



5(xf ) rQ(x|,) 



E 

>/3eff2(X,Z/rZ) 



e B/3,r(T, XL + X_r) 



we can deduce from (^) the following transformation law: 

Z,"(-l/r)=r-"(-V^r)-i2 



E 



e^Zf(.). 



This formula is of course compatible with Montonen-Olive duality for PSU{r) group [V-W]. 



(4.9) 



(4.10) 



(4.11) 



we set Xl"! = HilbJ. 



4.2. Relation to Theorem 3A. We shall check that Theorem OT is compatible with ( |4.8| ). For simplicity 



dZi i ^— , ax 

0<b<d 



Therefore we get 



J2 J2 E^^(^'"')'?''^"~^^9^'^^*'^^^'^^*"^^''^^^'''^^^"""^^^'^^^'''^^^ 

Q<b<d^eH^(X,Z) n 

E E d2^(X["l)g3("-i)g5aQ(«i)5^Q(?H)e''0(«^^) 

5Gff2(x,z) (4-12) 
E E d^x(^'^'"'^^^^^')9*^"''^9^'^*'''^"'^'^g^'^'''''"'^'"^e'''^^''''"'^'''-' 



E ^'^(^' 



[<«''>/2+l])g^((a™)^>gJ^Q(ci(a«;)i)--^Q(ci(au,)|)gQ(ci(Q«;),x)^ 



rk w—d 



In particular, if v is primitive, then by Corollary p.2| , we get 

"X(M(^;))" = x(^'<"'^/'+'l) = x(M(«)). 



(4.13) 



(4.14) 



This implies that x(-^(t')) is related to modular forms and in particular Hecke transforms. To understand 
the meaning of "x(M(t!))" for non-primitive ?; is a challenging problem. The relation to O'Grady's symplectic 
compactification of M(2 — 2llj) ([02]) is also an interesting problem. 



5. Appendix 



In this appendix, we shall explain our method for dimension counting of substacks of A^(u)'"'". Since 
most results are appeared in another forms (cf. [D-R], [H-N]), we only give an outline. 
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5.1. Notation. For an ample divisor H' on X, let Q{inH',v) be the open subscheme of the quot-scheme 
Quotc)^(_„^,)ffi]v/x/c consisting of points 

X:Ox{-mH')®^ ^ E (5.1) 

such that 

1. v{E) = V, 

2. A induces an isomorphism H°{X,0%^) = H°{X, E{mH')), 

3. H\X,E{mH')) = 0, i > 0. 

Let Oqi^mH' ,v)y.x{~'m.H')®^ — > Q„ be the universal quotient. We set Vy := Ox{-mH')®^ . For our 
purpose, the choice of mH is not so important. Hence we simply denote Q{mH' ,v) by Q{v). 

Let Qy : Q{v) M{v) be the natural map. We denote the pull-backs q~^{M{v)'^''-'^),q~^{M{v)'^''), . . . 
by . . . respectively. If we choose a suitable Q{v), then g„ : Q^v)'^"^ A^(v)^''* is surjective 

and M{v)'^'^'^ is a quotient stack of Q{v)'^'^'^ by a natural action of Gy := GL{N): 

M{vr'"^[Q{vr'yGy]. (5.2) 

From now on, we assume that : (3(u)'""' — > A^(u)'""' is surjective. 

5.2. Stack of filtrations. 

Definition 5.1. !F{vi,V2) is the stack of filtrations Fi C E, E E Ai{v) such that 

1. Fi is a /i-semi-stable sheaf of v{Fi) = ui. 

2. E'/i^i is a /x-semi-stable sheaf of w(£'/i^i) = W2- 

Let : 112) — > A^(w)^*'' be the projection sending {Fi C E) to E and p«i,t,2 : T{vi,V2) — > A^(wi)^^'* x 

M{v2)'"''' the morphism sending (Fi C E) to {Fi,E/Fi). 

We consider an open subscheme -F(wi,f2) of Quotg^/g^^-j^ss consisting of quotients {Qv)x ~* 
E2,x & (5(f )^'"' such that £^2 is a /i-semi-stable sheaf of v{E2) — V2- Then 

T{vi,V2)^[F{vuV2)/Gy]. (5.3) 

We shall give another expression of J-{vi,V2) which is useful to compute the dimensions of substacks of 
^{vi,V2) and its projections to M{v). 

We shah choose Q(mi/', w)^"", Q{mH' ,vi)'"'' and Q{mH',V2y for the same mi?'. Then Vy = Vy,®Vy^. 
For simphcity, we set Qi := Qy. ,Vi :— Vy., . . . and K-i are the universal subsheaves of Oq(^.)^ss (^Vt, i = 1,2. 
We define a scheme vj : Y Q(ui)^'''' x Q{v2Y^^ by 

y {V : (/C2)., ^ (Qi)xJ(a;i,:E2) e Q(«i)''^^ x Q(«2)''^^}. (5.4) 

Then Y parameterizes subsheaves K dV such that KOVi = (/Ci)xi and K/KOVi = {IC2)x2'- For a quotient 
V' : (^2)2:2 ^ (Qi)a;i, the subsheaf if of Vy is defined by 

K {(01,02) e V^i 0^2102 e (/C2):r2,?/'(a2) = aimod(/Ci)xJ. (5.5) 

Considering the quotient Vy — > Vy/K, Y also parameterizes the following exact and commutative diagram: 





(Qi).i > E > (Q2).2 ^ (5.6) 



> Vi > Vy > V2 > 

Let : Y X Gy ^ F{vi, V2) be the morphism sending {y : Vy E , g) E Y y. G2 to 

{yog:Vy-^E,E^Qx,)EF{v^,V2), (5.7) 

where ti7(y) = (xi,X2). Let P be the parabolic subgroup of Gy fixing Vi. Then Y has a natural action 
of P and ^ induces a morphism K Xp G„ — > P(di,7J2), which is G^-equi variant. It is easy to see that this 
morphism is an isomorphism (cf. [Y2, appendix]). Therefore 

^(^;i,t,2)-[rxpG,/G,] 
By using (^), we shall prove the following. 
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Lemma 5.1. We set 

Af''{vi,V2) ■■ = {iEi,E2) e M{vi)'''' X Miv2T'''\dimllom{Ei, E2) = n}, 

T-{v,,V2):^p-l,,{N'\v,,V2)) (5.9) 
= {{Fi ClE)£ J^{vi,V2)\dimUom{Fi,E/Fi) = n}. 

Then, 

diinT''{vi,V2) = dimAf''{vi,V2) + {VUV2) + n. (5.10) 

Proof. We set 

Q"(^;i,i;2) := {(xi,S2) G Q(^^i)'^''^ x Q(7;2)'^''1 dimHom((Qi),, , (Qs)..) = n}. (5.11) 

For (xi,X2) G Q"(wi,V2), there is an exact sequence 

^ Hom((Q2)x., (Qi).J ^ Honi(F2, (Qi),J Hom((/C2).,, (Qi).J ^ Exti((Q2)x., (Qi)xJ -> 0. 

(5.12) 

Since diniHom(F2, (21)2^1) = rkVirkV2 and dimExt^((Q2)a:2, (Qi)a:i) = Hom(/C2,Qi) is a locally free 
sheaf of rank (wi,i;2) + rkVirkV2 + n on Q"{vi,V2), where IC'2 and Q[ are pull-backs of JC2 and Qi to 
Q"(?;i,i'2) respectively. We set := V(Hom(/C^, Q'J^) ^ Q"(wi,i'2)- Then .f"(wi,W2) = [r"/P]. Hence 
we get that 

dim J^"(z;i, U2) = diniF" - dimP 

= dimQ"('Ui,i;2) + (wi,W2) + n - (diniGi +dimG2) (5.13) 
= dim A/"" (ui, 1^2) + (wi, -^2) + 

□ 



By similar method as in the proof of Lemma 5.1, we can also prove the following. 
Lemma 5.2. Let J-'^ivi, V2, ■ ■ ■ ,Vs) be the stack of filtrations 

d Fi d F2 d ■ ■ ■ d Fs = E, E e M{v) (5.14) 

such that 

1. Fi/Fi^i, 1 <i < s are semi-stable of v{Fi/ Fi^i) = vi. 

2. Hom(F,/J^,_i,i^j/Fj_i) - 0, i < ]. 
Then 

s 

dim («i , t;2 , . . . , ) - ^ X (i;.)'' + ^ 

{vi,Vj). (5.15) 

i=l i<j 



5.3. Supplement for the proof of Lemma 1.8. We shall explain how to derive (1.15) from (1.14). 
Let qi : Q{lwy'' MhQ-w) :— Qilwy^ / Giw be the quotient map. For a sequence of positive integers 
'1 < ^2 < • • • < of X]i=l — '1 ^6 

Mh{Iw; hM,---,ls)- = {®UiEf' e Mh{Iw)\Ei,E2, ...,E,e Mh{w), E, ^ E, for i ^ j}, 

Q{lw;li,l2, ■ ■ -Jsy ■■ = qi^{MH{lw]h,l2, ■ ■ -Js))- (5. 16) 

For simphcity, we set G := Giw and G; := G/.^, i = 1, 2, . . . , s. For quotients (/>i : 14 — > Q^^ & Q{kw, k)^" , 
i — 1, 2, . . . , s and an element g e G, we define a quotient 

i(BLl<P^)°9■■V^(BUQ..■ (5.17) 
It will define a morphism ni=i Qihw; x G — > Q{lw). Let Wl^i Q{liW, ky Xjj, q. G be the quotient of 
11^=1 Q{hw; X G by a natural action of Hi Gi- Then the above morphism induces a morphism 



7Ti,M,-,h ■■ n ^»)" G Q(/«;). (5.18) 

By the construction of 7Tij^^i2....,i^ , imTTij^i^,...,;, contains Q{l'W] Zi, ^2, ■ ■ • , ^s)*^- Hence 

s 

dim [g(?z«; /i, ?2, . . . , IsY'/G] < J2 dim [g(?»«;; hY'/G,] . (5.19) 

1=1 

We shall prove that 

dim[Q{kw;kYyG,]<l. (5.20) 
Then dim [g(Zu;;Zi,Z2,---,Z5)"VG] <s<l. Clearly dim [Q(/w; 1, 1, I)'* VG] = I. Hence dim[g(/u;)"VG] = 
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Proof of d^): Since qi- : Q{liw; liY^ — > M }i{liW] Ij) is surjective and AivaM }i{liW] li) — 2, it is sufficient 
to prove that 

dim [qi^\E®'^)/G,] <-l, EG Mh{w). (5.21) 
By definition, J{li,E) = [qi^^{E®^^)/Gi]. Hence we obtain this claim from (1.14). 

Acknowledgement. This paper heavily depends on Mukai's wonderful works [Mul,2,3]. I would like to 
thank Shigeru Mukai for valuable discussion on his works. I would also like to thank Toshiya Kawai for 
useful discussions on Jacobi forms. Main part of this work was done when I stayed at Max Planck Institut 
fiir Mathematik. I would like to thank Max Planck Institut fiir Mathematik for support and hospitality. 

References 

[Ba] Batyrev, V., Birational Calabi-Yau n-folds have equal Betti numbers, New trends in algebraic geometry (Warwick, 

1996), London Math. Soc. Lecture Note Ser., 264 Cambridge Univ. Press, 1999 
[B] Beauville, A., Varietes Kdhleriennes dont la premiere classe de Chern est nulle, J. Diff . Gcom. 18 (1983). 755—782 

[Br] Bridgeland, T., Equivalences of triangulated categories and Fourier- Mukai transforms, math. AG/9809114 

[De-L] Denef, J., Loeser, F., Germs of arcs on singular algebraic varieties and motivic integration. Invent, math. 135 

(1999), 201-232 

[D] Dijkgraaf, R., Instanton Strings and Hyperkdhler Geometry, NucL Phys. B 543 (1999), 545-571, hcp-th/981021C 

[D-R] Desale, U. V., Ramanan, S., Poincare polynomials of the variety of stable bundles, Math. Ann. 216 (1975), 233—244 

[D-L] Drezet, J.-M., Le-Potier, J., Fibres stables et fibres exceptionnels surP^, Ann. scient. Ec. Norm. Sup., 4"^ serie, t. 

18 (1985), 193-244 

[E-Z] Eichler, M., Zagier, D., The theory of Jacobi forms. Progress in Math. Vol. 55, Birkhauser, 1985 

[G61] Gottsche, L., The Betti numbers of the Hilbert scheme of points on a smooth projective surface. Math. Ann. 286 

(1990), 193-207 

[G62] Gottsche, L., Soergel, W., Perverse sheaves and the cohomology of Hilbert schemes of smooth algebraic surfaces. 

Math. Ann. 296 (1993), 235-245 
[G63] Gottsche, L., Theta functio ns and Hodge numb ers of moduli spaces of sheaves on rational surfaces. Comm. Math. 

Phys. 206 (1999), 105-136, |math.AG/9808007| 
[G-H] Gottsche, L., Huybrechts, D., Hodge numbers of moduli spaces of stable bundles on K3 surfaces, Intornat. J. Math. 

7 (1996), 359-372 

[G] Gieseker, D. On the moduli of vector bundles on an algebraic surface, Ann. of Math. 106 (1977), 45—60 

[H-N] Harder, G., Narasimhan, M. S., On the cohomology groups of moduli spaces of vector bundles on curves. Math. 

Ann. 212 (1975), 215-248 
[K] Kontsevich, M., Motivic integration. Lecture at Orsay, (7th Dec. 1995) 

[Mai] Maruyama, M., Moduli of stable sheaves H, J. Math. Kyoto Univ. 18 (1978), 557-614 

[Ma2] Maruyama, M., Moduli of algebraic vector bundles, in preparation 

[MNVW] Minahan, J. A., Nemeschansky, D., yafa. C. Warne r. N.P., E-Strings and N = A Topological Yang-Mills Theories, 

Nucl. Phys. B 527 (1998), 581-623 |hep-th/9802168| 
[M-W] Moore, G., Witten, E., Integration over the u-plane in Donaldson theory, hep-th/9709193 

[Mul] Mukai, S., Symplectic structure of the moduli space of sheaves on an abelian or K3 surface. Invent, math. 77 (1984), 

101-116 

[Mu2] Mukai, S., On the moduli space of bundles on K3 surfaces I, Vector bundles on Algebraic Varieties, Oxford, 1987, 

341-413 

[Mu3] Mukai, S., Moduli of vector bundles on K3 surfaces, and symplectic manifolds, Sugaku Expositions, 1 (1988), 

139-174 

[Ol] O'Grady, K., The weight-two Hodge structure of moduli spaces of sheaves on a K3 surface, J. Algebraic Geom., 6 

(1997), no. 4, 599-644 

[02] O'Grady, K., Desingularized moduli spaces of sheaves on a K3, J. reine angew. Mat h. 512 fl9991. 4 9-117 

[V-W] Vafa, C, Witten, E., A strong coupling test of S-duality, Nucl. Phys. B 431 (1994), |hep-th/9408074 



[Yl] Yoshioka, K., The Betti numbers of the moduli space of stable sheaves of rank 2 on P^ , J. reine angew. Math. 453 

(1994), 193-220 

[Y2] Yoshioka, K., The Betti numbers of the moduli space of stable sheaves of rank 2 on a ruled surface. Math. Ann. 

302 (1995), 519-540 

[Y3] Yoshioka, K., Ghamber structure of polarizations and the moduli of stable sheaves on a ruled surface. Internal. J. 

Math. 7 (1996), 411-431 

[Y4] Yoshioka, K., Betti numbers of moduli of stable sheaves on some surfaces, S-duality and mirror symmetry (Trieste, 

1995), Nuclear Phys. B Proc. Suppl., 46, (1996) 263-268 
[Y5] Yoshioka. K.. An application of exceptional bundles to the moduli of stable sheaves on a K3 surface, p.lg-| 



geom/9705027 Some examples of Mukai's reflections on K3 surfaces, (extended version) J. reine angew. Math. 



515 (1999), 97-123 

[Y6] Yoshioka, K., Euler characteristics of SU{2) instanton moduli spaces on rational elliptic surfaces. Comm. Math. 

Phys. 205 (1999), 501-517 
[Y7] Yoshioka, K., Moduli spaces of stable sheaves on abelian surfaces, preprint 

Department of mathematics, Faculty of Science, Kobe University, Kobe, 657, Japan 
E-mail address: yoshioka(Smath.kobe-u. ac . jp 



21 



